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Learning Outcomes in an online vs traditional course 


Abstract 

Relative enrollment in online classes has tripled over the last ten years, but the efficacy of learning online 
remains unclear. While two recent Meta analyses report higher exam grades for online vs. traditional classes, 
this body of research has been marked by two recurrent limitations: (1) a possible problem of selection bias 
wherein students self select the mode of course delivery and (2) a relative lack of proctoring of exams in 
online sections. Both of these confounders contribute to observed differences in performance. The present 
study addresses these limitations. Data refer to 64 students enrolled in criminology classes at a Carnegie 
research extensive university. Due to an administrative error in the course schedule, which failed to list one 
section as online, students were unable to self select into the online section, creating a rare opportunity for 
quasi randomization of students into sections. Both sections were taught by the same instructor. The 
dependent variable is the score on the standardized final examination. All exams were proctored by the 
instructor. The central independent variable is method of delivery of content: online vs. the traditional 
classroom. Controlling for other constructs, there was no significant difference between exam scores. Also, 
student evaluations did not differ between sections. Controlling for selection effects and the proctoring of 
exams, the academic performance of online students was the same as that of traditional students. Future work 
is needed for other courses, other fields, and other types of academic institutions. 
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On line instruction has been growing at a fast pace over the 
last decade. In 2002 a total of 1,602,970 students in higher 
education took at least one course online. By 2011 6,714,792 
students took one or more online classes. This changes amounts 
to an increase of 318.9%, or a 4.189 to one ratio. The prevalence 
of online instruction can also be measured as online enrollment as 
a percent of total enrollment. This percentage increased over 
three fold from 9.6% in fall 2002 to 32.0% in fall 2011(Allen & 
Seaman, 2013). This trend is illustrated in figure 1. Based on 
annual survey data from chief educational officials at up to 2,800 
institutions of higher education, these and other indicators of the 
prevalence of online instruction have tripled over the last decade 
(Allen & Seaman, 2013). Criminal justice programs have often 
provided leadership in the dissemination of online delivery of the 
curriculum. 

Figure 1. Trend in Percent of Students Taking at Least One Online 
Course in American Degree-Granting Colleges and Universities, 
2002-2011, all fields (Source: adapted from data in Allen & 
Seaman, 2013). 
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Learning Outcomes: online vs. traditional courses 


A recent development in online teaching is its extension to the 
MOOC. Free Massive Open Online Courses (MOOCs) have raised 
concern about the survival of higher education as we know it, a 
system centered on the traditional classroom delivery of 
knowledge. At present, 2.6% of higher education institutions 
have a MOOC while 9.4% report that they are in the planning 
stages (Allen & Seaman, 2013). Free MOOCS are often thought to 
be a means for ultimately recruiting tuition paying students. 
Flowever, there are reported difficulties in getting MOOC students 
to enroll on campus, and attracting students who will pay a fee to 
take a MOOC for actual college credit (Kolowich, 2013). At 
present, research on learning outcomes of online vs. traditional 
classes has not rigorously assessed MOOCS. There is a substantial 
literature, however, on perceptions and analyses concerning 
student learning outcomes in online vs. traditional classes. 

Given the increasing use of online instruction, it is important 
to assess the learning outcomes of students enrolled in online vs. 
traditional classes. In terms of perceptions, the opinions of chief 
academic officers at nearly 3,000 colleges are split on the extent 
to which student achievement is the same, higher, or lower in 
online vs. traditional classes. Flowever, the greater the 
involvement of a college in online learning, the higher the 
probability that its chief academic officer believes that students 
learn more in online vs. traditional classes (Allen & Seaman, 

2013). Whether or not the perceptions of higher education officials 
reflect reality is subject to a review of the quantitative work 
comparing grades achieved in online vs. traditional classes. 

Previous research that rigorously compares student 
achievement between online and traditional classes is marked by 
some conflicting findings as well as some recurrent limitations 
(Bray, Flarris & Major, 2007; Figlio, Rush, & Yin, 2010; Gratton- 
LaVoie, 2009; Flarmon, 2006; Brown & Leidholm, 2002; Parsons- 
Pollard, Lacks & Grant, 2008; for reviews see Means, Toyama, 
Murphy, et al. 2010; Shachar & Neumann, 2003). First, in nearly 
all studies, students can freely select to enroll in online vs. 
traditional classes. To the extent that the characteristics of online 


https://doi.org/! 0.20429Hjsotl.2015.090105 


2 



IJ-SoTL, Vol. 9 [2015], No. 1, Art. 5 


students differ from their traditional counterparts, in terms of such 
characteristics as GPA, age, marital status, maturity, and learning 
styles such as audio vs. visual learning, self selection can bias the 
results on academic achievement (e.g., Allen & Seaman, 2013; 
Bray, Harris & Major, 2007). Second, the procedures involved in 
the measurement of student achievement are largely unclear. To 
the extent that examinations are given online with little or no 
supervision, the achievement of students in online classes may be 
greater than in traditional classes. In traditional sections 
examinations are supervised, thus minimizing cheating and 
collaboration in test taking. There is evidence that the incidence of 
overall cheating (including cheating on exams, papers, and other 
modalities of evaluation) in online classes is up to four times 
greater than that in traditional classes (Lanier, 2006; Moten, 
Fitterer, Brazier, Leonard & Brown, 2013). To the extent that 
cheating is more prevalent in online instruction than traditional 
instruction, reported differences between groups in student 
achievement need to be interpreted with caution. 

The present study contributes to the literature by addressing 
these limitations. First, it was able to inadvertently minimize 
opportunities for self selection into the online section of the 
course. Due to an administrative error in the schedule of classes, 
the online section was advertised as a traditional class. This 
feature of the study is relatively unique. It allows for controlling, 
at least in part, differences in learning styles and motivations, 
among the students in online and traditional sections of the same 
course. Second, it controls for the testing environment by 
proctoring exams on campus for both online and traditional 
sections of the course. Third, no course paper was required in 
any section, thus removing opportunities for cheating on that 
potential modality of learning outcomes. Finally, unlike some 
previous studies, the online and traditional classes were taught by 
the same instructor, thus minimizing instructor effects on 
achievement. 

The present investigation will review the literature on 
student achievement in online vs. traditional classes. Some special 
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attention will be drawn to student achievement in criminology 
classes since the present study focuses on classes in that field. 

The investigation then will contribute new findings to the literature 
by performing one of the first studies close to a case-control 
design, minimizing self selection effects. This will be the first such 
study for the field of criminology. 

LITERATURE REVIEW: ONLINE VS. TRADITIONAL STUDENT 

ACHIEVEMENT 

While there are a large number of investigations on the 
possible impact of instruction online vs. traditional classes on 
student achievement, there are conflicting findings(e.g., Bray, 
Harris & Major, 2007; Figlio, Rush & Yin, 2010; Gratton-LaVoie, 
2009; Harmon, 2006; Brown & Leidholm, 2002; Parsons-Pollard, 
Lacks & Grant, 2008; Russell, 1999; for analytical reviews of 86 
and 50 studies respectively see Means, Toyama, Murphy, et al. 
2010; Shachar & Neumann, 2003). Some investigations report 
that exam scores are higher for traditional classes than online 
classes (e.g., Brown & Leidholm, 2002; Figlio, Rush & Yin, 2010; 
Parsons-Pollard, Lacks & Grant, 2008) while others report the 
reverse, that student performance is higher for online sections 
(e.g., Gratton-LaVoie, 2009; Harmon, 2006; Means, Toyama, 
Murphy, et al., 2010). Still others report no significance difference 
in student performance between online and live classes (for a 
review see Russell, 1999). Caution needs to be exercised in 
interpreting the findings in this body of research for a series of 
methodological limitations. For example, some research compares 
online classes with traditional classes taught by different 
instructors. In such a research design observed differences may 
be largely due to teacher effects rather than mode of delivery 
effects (Brown & Leidholm, 2002). Online classes are thought to 
provide more opportunities for cheating, a behavior that can 
enhance student performance. Available survey data indicate a 
higher self reported instance of cheating in on line classes relative 
to traditional classroom based sections (Lanier, 2006; Morton, 
Fitterer, Brazier, Leonard & Brown, 2013 ). 
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A Meta-analysis of 86 studies determined that students in 
online sections of a course generally score higher on standardized 
final exams than students enrolled in traditional classes (Shachar 
& Neumann, 2003). The reported difference was large, amounting 
to a half of a standard deviation. A more recent Meta analysis, 
limited to 50 findings from the relevant research, also confirmed 
that academic performance was higher in online vs. traditional 
classes (Means, Toyama, Murphy, et al., 2010). However, there is 
a wide variety of confounders that may artificially enhance student 
performance in online classes. Most research was unable to or did 
not control for factors which may give students in online classes 
the edge over their traditional counterparts in exam scores. These 
factors include two which provide a focus for the present 
investigation: (1) selection bias in choice of mode of delivery 
(online vs. traditional), and (2) opportunities for cheating with a 
focus on the extent to which exams are proctored. 

Most research has been unable to control for possible 
selection bias given such issues as practical barriers in randomly 
assigning students to online vs. traditional sections of a given 
course (for an exception see Figlio et al., 2010), and the 
unavailability of complete data on the background characteristics 
of students (Bray, Harris & Major, 2007). Students who freely 
choose online classes may have different characteristics than 
students who choose traditional, live classes. For example, 
students opting for online classes may be older, have children, 
and/or be fully employed (Bray, Harris & Major, 2007). Online 
classes can be attractive to such groups since they minimize 
commuting time and can reduce or eliminate the need for child 
care when studying course material. Online classes can resolve 
conflicts between work and schooling since online class material 
can be studied at night, on the weekends, and other times during 
non-work hours. 

For example, a study of learning outcomes (exam scores) in 
online vs. traditional classes in microeconomics determined that 
students in the online class scored higher on the final exam than 
the traditional class (68.1% vs. 61.6%). However, the classes, 
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online vs. traditional, differed significantly in the kinds of students 
who chose each respective mode of delivery. For example the 
online students were older (25.3 vs. 20.7 years), more apt to be 
married (29% vs. 6%), to have children (21% vs. 4%), had a 
higher GPA (2.85 vs. 2.57), and to have taken a previous 
economics course (59 vs. 40%). Once these differences between 
groups in various background characteristics were controlled, 
there was no significant difference in exam scores (Gratton-Lavoie 
& Stanley, 2009). In order to fully control for background 
characteristics, a randomized case-control research design has 
been advocated (Bray, Harris, & Major, 2007). The present study 
addresses this call through a control for self selection. 

Most research does not report the presence or degree of 
proctoring exams. The absence of a proctor during exams 
increases opportunities for cheating. While there have been 
technologies developed to reduce cheating, such as having 
students show ID's while taking exams on a webcam, it is not 
clear if these have been enough to reduce cheating. Students 
report that they are up to four times more likely to cheat in online 
classes compared to traditional classes (e.g., Moten et al., 2013). 
Traditional classroom instruction generally involves the presence 
of a proctor during exams. This generally assumed not to be the 
case in online classes. Research on online instruction often does 
not report the details of the online examination environment (e.g., 
Brown & Leidholm, 2002). 

That the presence or absence of a proctor makes a difference is 
demonstrated in a study of online vs. traditional classes in 
introductory economics. Online students who were able to take 
the exams without the presence of a proctor did, on average, one 
letter grade better than online students whose exams were 
proctored (Wachenheim, 2009). Intuitively, this would be 
expected since the absence of a proctor can entail an "open book" 
exam, which can give the unproctored students an advantage 
over the students taking the exam with a proctor (presumably 
closed book). This may help explain the finding that online 
students tend to do better than their counterparts in traditional 
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classes traditional (Means et al., 2010; Shachar & Neumann, 
2003). 

There has been little research on the problem in the specific 
field of criminology. Only one previous relevant investigation was 
found. Parsons-Pollard, et al. (2008) assessed differences in 
student achievement in sections of introduction to criminal justice. 
The sample was based on 305 students in a large traditional 
section and 425 students in a large online section of the course. 
Students in the traditional section received significantly higher 
final grades than students in the online section, but the difference 
was not large (81% vs. 78%). Caution needs to be exercised in 
interpreting the results of the Parsons-Pollard, et al. (2008) study. 
The examinations were not exactly the same between sections, 
although they are reportedly "similar." Differences in the rigor of 
the exams might explain the reported differences in mean grades. 
It is not reported if the sections were taught by the same faculty 
member, so that teacher effects on learning may be present. 
Students self selected the online vs. traditional sections, opening 
up the possibility of selection effects. The examination 
environment is not described, so that it is not known if there were 
differences in the proctoring of traditional and online sections. 

Still, the direction of differences (students in the traditional 
section performed better than online students), runs counter to 
the conclusion of two meta-analytic reviews (Means et al., 2010; 
Shachar & Neumann, 2003 ). Further work is needed to address 
this issue. 

The present analysis is able to address two limitations of 
previous work: selection effects and differences in the degree of 
proctoring of exams between groups. The present study was able 
to, in effect, quasi randomize students into online vs. traditional 
sections of the course. Second, it controls for the presence of a 
proctor by arranging for a proctor (the instructor) during exams in 
both the online and traditional sections of the course. It also 
contributes the first study of its kind for the field of criminology. 

METHODOLOGY 
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Subjects were all students enrolled in two sections of 
criminological theory at a Carnegie research extensive university 
during fall 2005. Due to an error in the printing of the course 
schedule by the office of scheduling, neither class was defined as 
an online class. However, one was online and the other was a 
traditional live class. The students were all surprised in the online 
section when they found out it was an online course (the 
instructor was also surprised since it was the only time such an 
error appeared in course scheduling, and has never been the case 
since). Importantly, there was no differential exodus from the 
online section once the students found out it would be on line. Of 
the 33 students enrolled in the online section, only one dropped. 

Of the 34 students who enrolled in the traditional live class, only 
two dropped the course. Hence, complete data were available for 
32 students in the online section and 32 in the traditional offline 
section. 

The scheduling error resulted in a research design 
approaching that of a randomized clinical trial. As far as the 
author has been able to determine in a review of published 
research in refereed journals, the present analysis is the one that 
comes closest to randomizing students to an online and traditional 
section of the same course. This has the advantage of minimizing 
selection effects such as a common view that online classes 
attract and are best for the more disciplined students or for visual 
learners over auditory learners who thrive on instructional 
modalities such as classroom based, live discussions (e.g., Allen & 
Seaman, 2013). 

The classes had exactly the same reading assignments and 
examinations. The power point slides (N = l,400) used in the 
traditional class, together with accompanying audio files of lecture 
in the traditional class, were required materials to be reviewed by 
members of the online section. There were opportunities for 
discussion in both classes, but the modalities were different. The 
traditional class had opportunities for questions from the students 
and subsequent discussions. The online class had a discussion 
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board to facilitate discussion of the course material, but 
participation, as in the traditional class, was voluntary. 

The principle dependent variable is the score on the final 
examination. Importantly, the exams, including the final exam, 
were given under supervised conditions on campus. Online 
students were called back to campus for exams. The instructor 
was present to proctor all exams for both the online and 
traditional sections. This minimized opportunities for cheating on 
exams, removing one of the potential sources for measurement 
error in the past research (Lanier, 2006). 

The central independent variable is a binary variable, type of 
course delivery system (0,1). Delivery is coded where l=online 
delivery and 0= the traditional class. 

Control Variables. Grades on the first hour exam are used as 
a proxy independent variable for several constructs thought to 
predict student achievement. These constructs include academic 
ability, amount of academic effort, and the amount of time spent 
and/or available for studying course material (Stack, 2013). Data 
on these specific constructs were unavailable, but it is assumed 
they are at least partially captured by grades on the first hour 
exam. In results not fully reported here, an average of the first 
two hour exams was used as the proxy measure of omitted 
variables. However, the results were essentially the same. In 
addition, a control is included for the gender of the student where 
l=female and 0 = male. 

A second dependent variable to be analyzed is student 
evaluation of instruction. Student evaluation (SET) data is from 
the standard university SET forms. Each of three summary 
measures is employed. These are the three that are emphasized 
by the university administration: (1) How would you rate this 
course, (2) How much have you learned in this course, and (3) 

How would you rate the instructor's teaching in this course? 
Responses are based on a five point scale where l = poor/nothing 
through 5= excellent/a great deal. Each subscale is analyzed 
separately. 
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ANALYSIS 

Table 1 provides the mean scores on the variables for the 
traditional class vs. the online class. In preliminary results not 
fully reported here, Pearson correlation coefficients were 
calculated. Type of delivery system (offline vs. online) was 
unrelated to final exam scores (r 0.147, p > .05). Type of course 
delivery system was also unrelated to scores on the first hour 
exam (r 0.201, p >.05) and the second hour exam (r 0.105, p > 
.05). 

Table 1. Variable Means in the Traditional vs. Online Sections of 
Criminological Theory. 


Variable 

Mean, 

Traditional, 

Live 

Section 
(N = 32) 

Mean, 

Online 

Section 
(N = 32) 

Examination 

67.5 

72.1 

1 



Final Exam 

56.6 

59.9 

Gender 

1.48 

1.56 

(Female=2) 




The results of the multivariate analysis are provided in Table 
2. Controlling for the other predictors, students in the online 
course delivery system did no better on the final exam than the 
students in the traditional class (bl.14, p >.05). The coefficient 
for the online course variable was only 0.46 times its standard 
error. Grades on the first hour exam predicted final exam scores 
(b .515, p < .05). The coefficient for first hour exam grades was 
4.74 times its standard error. Gender was unrelated to final exam 
scores (b -.217, p > .05). The model as a whole significantly 
predicted final exam scores (F 8.51, p < .05). From the R 
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squared statistic, the model explains 29.5% of the variance in 
final exam scores. 


Table 2. The Effect of Mode of Delivery (Online Vs. Traditional 
Classes) on Student Achievement, First Hour Exam, and Gender 
on the Final Exam in Criminological Theory (N = 64 students). 


Variable 

Regression 

Coefficient 

Standard 

Error 

t- 

testest 

Mode of 
Delivery 
(Online 

Class,0,1) 

1.14 

2.48 

0.462 

Grade, 

First Hour 

Exam 

.515* 

.109 

4.76 

Gender 

(female= 

1) 

-2.17 

2.43 

bo 

VO 

Constant 

24.93* 

8.55 

2.91 

F-Statistic 

8.50* 



R- 

squared 

.295 




* p < .05 




Table 3 provides the results on the relationship between 
method of course delivery (traditional vs. online) and student 
perceptions of instruction. The overall student perception of the 
course did not differ by mode of delivery. This mean rating was 
2.4 in both the traditional and online sections. The SET's were also 
identical on the perception of the amount learned. Students 
reported a rating of 2.8 in each section. The item: "How would 
you rate the instructor's teaching in this course?" received largely 
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the same mean score in each section, 2.9 in the traditional class 
and 2.8 in the online class. These SET scores are relatively low, 
but may be attributed to the instructor's grading policy. Available 
data indicate that the mean course grades of the students in both 
classes are relatively low for the department. 


Table 3. Mean Student Evaluation of Teaching (SET) Scores in the 
Traditional vs. Online Sections of Criminological Theory (N = 64 
students). 


Summary SET 

Item 

Traditional, 

Live Section 
(N = 32) 

Online 

Section 
(N = 32) 

How would you 
rate this course? 

(l = poor through 
5=excellent) 

2.4 

2.4 

How Much have 
you learned in 
this course? 

(l = nothing 
through 5=a 
great deal) 

2.8 

2.8 

How would you 
rate the 

instructor's 
teaching in this 
course? (l = poor 
through 

5=excellent) 

2.9 

2.8 


CONCLUSIONS 

Online education as a means of course delivery has 
proliferated in the last decade. Both the number of students 
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taking online classes and the proportion of online classes of all 
classes have more than tripled (Allen & Seaman, 2013). While 
there is evidence that the achievement of online students is 
significantly higher than the achievement in classes taught in 
traditional classes (for reviews see Means, et al., 2010; Shachar 
& Newman, 2003), the interpretation of this finding is open to 
some question. Many previous studies were not able to control for 
selection effects. It is plausible that the more industrious, married, 
mature, older, self motivated students are more apt to select 
online classes than their counterparts as was found in a study of 
microeconomics classes (Gratton-Lavoie & Stanley, 2009). Bray, 
Harris & Major (2007) call for investigations that randomize 
students into online vs. offline sections. The present study 
answers this call. 

The results of the present study find that there is not a 
significant difference in the final exam scores of the students in 
online and traditional sections of the course. Previous work has 
been largely unable to randomize students into experimental and 
control groups, online vs. traditional sections. In contrast, the 
present study, due to an administrative error, was able to 
essentially, randomize students into online and traditional sections 
of criminological theory. In the previous body of research, the 
lower achievement levels of students in traditional classes may 
simply be an artifact of selection effects. 

In addition, given that online students have more 
opportunities for cheating and report up to four times more 
cheating in their online classes than their traditional classes, the 
higher achievement levels of online students may be subject to 
measurement error (Lanier, 2006; Moten et al., 2013). On line 
students often have advantages on exams including taking exams 
on an open book environment and with the help of other students 
in unsupervised environments. The present study minimized 
opportunities for cheating on exams by calling back the online 
students to campus to take all exams in a supervised 
environment. This element of controlling for opportunities for 
cheating also helps to explain the similarity of test scores between 
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online and traditional classes in the present investigation. In 
previous research the reported gap between learning outcomes in 
traditional vs. online classes may be, in part, an artifact of 
corresponding differences in the level of cheating. 

Given its quasi randomization into experimental and control 
groups, and minimizing opportunities for cheating, the present 
study improves on previous research. It questions the reported 
higher levels of learning among online students found in previous 
work (Means et al., 2010; Shachar & Neumann, 2003). Given the 
rapid spread of online learning, future work is needed to control 
for these issues in other fields and other types of academic 
institutions. 

The finding of no difference in SET scores between online 
and traditional classes is largely consistent with previous work on 
this issue (for a review see Parsons-Pollard, et al., 2008). 

However, in some previous research, students give online classes 
a lower rating, apparently due, in part, to malfunctions in online 
technology systems during examinations (Parsons-Pollard et al., 
2008). 

There has been only one previous similar study for the field 
of criminology. The results on student achievement in the present 
study contrast with the previous investigation. Parsons-Pollard, et 
al. (2008) reported that the grades received by students in a large 
traditional introductory level criminal justice class were 
significantly better than the online students. However, the 
difference, while significant, was small. The means were 81% vs. 
78%. Nevertheless, some methodological differences between the 
present study and that study might help to explain the different 
results. For example, students could self select the method of 
delivery, most of the students were not criminal justice majors, 
the same exams were not given across sections, and details are 
lacking on possible differences in the proctoring of exams. 

Finally, future research is needed on related issues beyond 
the scope of the present study. Retention and graduation rates 
may be related to mode of delivery. Perhaps traditional classes 
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might increase retention and graduation rates through face to face 
contacts between students, and between students and their 
professors. 

The goal of maximizing learning through online delivery may 
be compromised in the interest of cost-cutting. It is not fully clear 
what drives the trend towards online instruction. Online 
instruction is often driven by a quest to cut costs. For example, 
monies can be saved when classrooms are not needed for 
instruction. It is also unclear if a quest after cutting costs through 
online instruction affects staffing decisions. At the present 
institution online instruction in CJ, and some other departments, is 
almost entirely done by part time faculty. To the extent that full 
time faculty are more capable instructors than part time faculty, it 
is important that full time faculty are represented proportionally 
in online instruction. An over-reliance on part time faculty for 
online sections, a pattern that lowers labor costs, may be 
associated with falls in student achievement, retention, and 
graduation rates. 

Taken to the extreme, online instruction could be delivered 
through MOOCs centered at a relatively few traditional colleges 
and universities. Some experts suggest that in half a century 
there will only be 10 universities in the world, the ones that 
produce the MOOCS for a global audience. There are powerful 
social and economic forces involved in the trend towards MOOC 
delivery systems. The emergence of massive open online classes 
(MOOCs) has generally involved partnerships between 
corporations and non profit organizations on the one hand and 
universities on the other hand. MOOCs have spread through 
geographic space. The corporate/non profit developers include 
Coursera, Udacity, and edX in the US, Open2Study (Australia), 
FutureLearn (Britain), iversity (Germany), and Veduca (Brazil). 

Recently, there has been some fall in concern concerning 
the extent to which MOOCs might replace university based online 
classes. Thus far, MOOCS have generally been offered for free 
and without college credits. An experiment in the fall of 2012 at 
Colorado State University-Global offered a MOOC for credit, 
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charging only $89, the price of proctoring the final exam. 

However, no one registered for the course. This suggests that the 
audience for MOOCs is composed of persons not seeking college 
credit such as persons who already have college degrees, retirees, 
the curious who lack motivation for completing the course, and 
persons interested in only a subsection of the course. 

Nevertheless, corporations and non profit organizations are 
reportedly increasing efforts at advertising MOOCs in a quest to 
attract paying subscribers (Grossman, 2013; Kolowich, 2013). The 
extent to which MOOCs are a threat to traditional universities 
remains unclear. 

Finally, future research needs to take into account "ways of 
teaching" online and traditional classes. For example, there are a 
number of strategies to improve the quality of student learning in 
traditional courses. These strategies include active learning, 
educative assessment, the use of small groups, and using the 
taxonomy of significant learning to define learning outcomes. 
Transporting such techniques from the traditional classroom to the 
online environment can present something of a challenge for 
future research. 
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